• a computer with internet connection
• an account on super-computing Wales
• basic knowledge of DNA and genome structure

Lecture slides

https://drabarlow.github.io/bioinformatics_bootcamp/

Bootcamp worksheet

https://drabarlow.github.io/bioinformatics_bootcamp/bootcamp_worksheet_2025.html

Github repo

https://github.com/drabarlow/bioinformatics_bootcamp

Supercomputing Wales shared project

• please request access to: scw2141 Bioinformatics training

Axel Barlow, Lecturer in Zoology at Bangor University

• Self taught post-PhD
• No proper bioinformatics/computer science background
• Knowledge of bash and R.

Johanna Paijmans, Lecturer in Zoology at Bangor University

• Self taught (rebelling against a Linux-expert dad)
• No proper bioinformatics/computer science background
• Knowledge of bash and R.

• Released in 2016
• Allows installation of Linux distributions (e.g. Ubuntu) within Windows 10/11
• Provides a bash terminal and Linux environment
• Transfer between Windows and Linux filesystems possible
• Essentially WSL is Microsoft’s admittance to Linux superiority

*** =left

• a shell allows users to execute OS tools
• Accessed using a terminal
• Unix terminal came with the Bourne shell (sh), developed by Steven Bourne in 1979
• In 1979 Brian Fox in improved version: the Bourne again shell (bash)
• Most Unix-like OS use bash or something like it
  • execute standard OS functions and installed programs
  • access filesystem
  • supports bash scripts
  • pipes, auto-completion, loops, wildcards, etc.

*** =right

• Computational nodes
  • >134x Intel nodes with 40 cpus + 192 Gb RAM each
  • >64x AMD nodes with 64 cpus + 256 Gb RAM each
  • >26x Highmem nodes with 384 Gb RAM each
  • >28x Nvidia GPU nodes
• Storage space
  • 1192TB (usable) scratch space
  • 420TB of home directory space
• Access
  • Multi-factor authentication (MFA)
  • SSH jump host

• Scientific Linux OS
  • Command-line only (ssh)
  • File transfer using scp
• Space & quota
  • Home directory (small, persistent): 50Gb
  • Scratch space (bigger, temporary): SCW projects, few Tb
  • Max 10 active jobs, max 30 in the queue
  • Max 3 day runtime

• Command-line only (via ssh)
• Access through SSH jump host
• File transfer using scp or sftp
• Job submission using slurm job scheduler
• Software installed as modules
• No super user access
• Space & quota
  • Home directory (small, persistent): 50Gb
  • Scratch space (bigger, temporary): SCW projects, few Tb
  • Max 10 active jobs, max 30 in the queue
  • Max 3 day runtime

Connecting to the jump host (with MFA)

ssh you25usr@ssh.bangor.ac.uk

Note: most UNIX systems do not show anything when you’re typing your password!

If successful, connecting to Hawk

ssh b.you25usr@hawklogin.cf.ac.uk

Raise your hand if you are having issues 🙌

• AI tools are becoming increasingly common in bioinformatics (and everywhere else)

• It’s incredible powerful, but keep various considerations in mind (ethics, reproducibility, accuracy, privacy…)

• Different models are better or worse for different tasks (e.g. I find ChatGPT 4 is not great at programming)

• / [root] is uppermost level of filesystem
• Everything is contained in /
• Directories exist within the filesystem, they can contain files and other directories
• We specify a path through this hierarchy using forward-slashes
• Our current directory is called the working directory

/home/b.xlb21brx/
/scratch/b.xlb21brx/

• We can navigate through the filesystem (change working directory)
• Or we can specify the patch to directories or files remotely

• Simple Linux Utility for Resource Management: slurm
• Free open source job scheduler for linux systems
• Used on 60% of World’s top 500 computers
• Assigns user jobs to computer resources
• Submit to queue
• Short, low-resource jobs move faster through the queue
• Useful tools for scheduling, reporting, etc

• The current market leader
• Massive output
• Many applications (genome resequencing, RADseq, transcriptomes, metabarcoding)
• Cheap (£10 per Gb)
• Major limitation is the read length

*Indexes allow multiple samples to be sequenced at the same time

[Not an exhaustive list]

• Whole genome sequencing (pure DNA sample from a single individual)
• Reduced representation genome data (RADseq, targeted SNPs, single individual)
• Poolseq (multiple individuals)
• Transcriptome (RNA sample from single tissue/individual)
• Metabarcoding (PCR amplicon, multiple individuals/species)
• Metagenomics (whole genomes, multiple individuals/species)

Short reads from a single individual can be mapped to a reference genome assembly

• Nikolas Basler, Achim Klittich, Axel Barlow
• An environment for organising, processing, and archiving Illumina data
• BEARCAVE philosophy
  • All users can access all data
  • Avoid data redundancy
  • All samples processed using identical software programs and parameters
  • Incorporates sample metadata
  • Documents results of data processing
  • Easy to use wrapper scripts for programs
  • Publicly available
  • Safeguards in place to ensure consistency
• Consequently BEARCAVE is not for everyone, and has idiosyncrasies in use

sample|locality | adder01-04|Dublin adder05-08|Belfast adder09-12|Cork adder13-16|Limerick adder17-20|Galway adder21-24|Dundalk adder25-27|Bray adder28|outgroup

• fastq is the standard output format for data from Illumina (and other) platforms

@A00551:758:HKTVJDSX7:4:1101:3595:6872 1:N:0:CCTGAGATGT+GGTCTAGTTG
CTGAATATGGATTTTAATTGAATCCTAAGATATTATAGCATCTTTCACTCCCTGTCCTGTGCATGTCAGA
+
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

• Line 1: info on sequencer, flowcell, cluster position, indexes (sometimes)
• Line 2: called bases
• Line 3: a +
• Line 4: quality scores on Phred scale
• 10 = 90% accuracy; 20 = 99% accuracy; 30 = 99.9% accuracy
• Recoded as single character: F = 37; ? = 30; 5 = 20; + = 10

• DNA can be fragmented
• The fragment lengths have a distribution

45 ka cave bear (Ursus kudarensis)

• decompress fastqs
• trim adapter seqs using Cutadapt
  • 30 bp min length
  • min overlap 1 bp
• merge overlapping read pair using FLASH
• recompress files and clean up
• save appropriate log files

Expected output in /BEARCAVE2/trimdata/*processing/

• merged reads *_mappable.fastq.gz [big file]
• unmerged R1 *_mappable_R1.fastq.gz [big file]
• unmerged R2 *_mappable_R2.fastq.gz [big file]
• trim report *_trim_report.log and merge report *_merge_report.log

• decompress fastqs
• merged (SE) and PE data processed separately
  • mapping using bwa mem algorithm
  • PCR duplicates identified and removed using samtools
  • Reads with poor mapping quality (Q30) removed using samtools
• SE and PE data merged
• mapping log file generated
• file cleanup and renaming

Expected output in /BEARCAVE2/mapped*/*processing/

• mapped filtered data *.bam [big file]
• bam index *.bam.bai
• mapping log *_mapping.log

• Widely used program
• MANY population genetics analyses possible
• Tends to work directly from bam files (unlike plink, admixtools, etc)
• MANY filters available
• Genotype likelihood approach is a particular speciality

Allele1|Allele2|prob11|prob12|prob22 |||| A|T|0.05|0.9|0.05

• Several spin off programs that use GLs
  • NGSadmix
  • PCangsd
  • NGSrelate
  • realSFS

Covariance matrix

• All indviduals
• Allele frequency covariance matrix
• Used for PCA

Distance matrix

• All individuals
• absolute genetic distance between populations
• used as input for NJ algorithm

Heterozygosity

• Your single adder
• Calculate GLs, ML estimation of SFS along a sliding window using realSFS

• GUI for R
• Preferred by many
• Linked to tidyverse
• Linked to R markdown
• Version control (git) and other development tools

Most people disagree (in some cases strongly)

• Base R is really good!
• tidyverse is not the way to teach R to beginners
• ggplot2 code is restrictive
• It’s boring if everyone’s graphs look the same
• It’s OK to type commands directly into the terminal
• It’s OK to modify a raw data file
• It’s OK to combine other tools and languages
• Data doesn’t have to be tidy

Objects

• Contain data and results
• Created with <-
• Stored in active memory
• Names include letters numbers _ .
• Names must begin with a letter

Functions

• Carry out operations on objects
• Often generates new objects
• function()
• ?function

Vector

• List of values of the same type
• Numbers, strings, or logical values
• Can be generated using c()
• Indexing vector objects my_vector[]

Matrix

• 2D data of same type in rows and columns
• Indexing matrix objects my_matrix[row, column]

Dataframe

• Rectangular “table”
• Mixture of data types
• Set of vectors of equal length
• Extract columns with $, which can then be indexed like vectors

List

• Set of components with different structures
• Extract named components with $

• Dynamic documents in Rstudio
• Combine text, code, and results
• Publishers are increasingly asking for reproducible code capsules